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In reply to Ref.[l|] we demonstrate why the arguments made therein do not address the criticism 
exposed in Ref. 0] on the fundamental shortcomings of the Bayesian approach when it comes to 
the extraction of parameters of Nature from experimental data. As for the isospin analysis and 
the CKM angle a it is shown that the use of uniform priors for the observed quantities in the 
Explicit Solution parametrization is equivalent to a frequentist construction resulting from a change 
of variables, and thus relies neither on prior PDFs nor on Bayes' theorem. This procedure provides 
in this particular case results that are similar to the Confidence Level approach, but the treatment of 
mirror solutions remains incorrect and it is far from being general. In a second part it is shown that 
important differences subsist between the Bayesian and frequentist approaches, when following the 
proposal of Ref. [l[ and inserting additional information on the hadronic amplitudes beyond isospin 
invariance. In particular the frequentist result preserves the exact degeneracy that is expected from 
the remaining symmetries of the problem while the Bayesian procedure does not. Moreover, in the 
Bayesian approach reducing inference to the 68% or 95% credible interval is a misconception of the 
meaning of the posterior PDF, which in turn implies that the significant dependence of the latter 
to the chosen parametrization cannot be viewed as a minor effect, contrary to the claim in Ref. 



I. INTRODUCTION 

In Ref. [2| we have shown through the example of the 
extraction of the Cabibbo-Kobayashi-Maskawa (CKM) 
angle a from B — > 7T7t, pp decays that the Bayesian treat- 
ment as done, e.g., by the UTfit collaboration Q, suffers 
from major difficulties. The problems we have found are 
related to the presence of unknown free parameters which 
should actually be constrained by the data. We have 
shown that the results of the Bayesian analysis depend 
on the priors and the chosen parameterization in an un- 
controllable manner, and may even diverge in some cases. 
Furthermore we have demonstrated that the priors can- 
not always be specified in a consistent way with respect 
to the symmetries of the problem, which results in the 
present example in an incorrect description of the a — * 
limit. 

The authors of Ref. [l[ replied to our criticism. In the 
first part of their paper they agree with us on the de- 
pendence of the analysis with respect to the choice of 
priors and parameterization, even when the data are suf- 
ficiently precise so that the posterior PDFs are contained 
in the corresponding prior ranges. Ref. [l[ claims that 
this should not be viewed as a fundamental drawback of 
the Bayesian treatment, because in the absence of addi- 
tional information on the parameters one should use uni- 
form priors on the quantities that are directly measured 
(what we have called the "Explicit Solution" parameter- 
ization in [2|). We emphasize in the following that this 
recipe can only be (approximately) justified within the 
framework of classical statistics and is thus not actually 



Bayesian. Moreover, it is far from being general. 

In the second part of Ref. [1] it is argued that in ad- 
dition to the isospin analysis one may use bounds on the 
magnitude of the fit parameters related to hadronic ma- 
trix elements, and when taken into account the Bayesian 
treatment behaves more reliably. While this new rec- 
ommendation may appear physically sound, theproblem 
reported in Q uses the original isospin analysis Q , where 
the hadronic parameters are unknown and only isospin 
symmetry is used. Changing the hypotheses cannot be 
considered as a satisfying answer to our findings: indeed 
the validity of the statistical approach must be proven 
independently of the problem at hand. Despite this in- 
consistency, we accept the new proposal and show below 
that even with the use of this additional information one 
finds important quantitative and qualitative differences 
between the frequentist and the Bayesian results. 

In any case, the new recommendations of the UTfit 
collaboration for treating problems with a priori free 
unknown parameters differ quite importantly from their 
previous publications, which implies that the results pub- 
lished therein are obsolete. We are therefore expecting 
new analysis results for the angles a from B — » pn and 
7 from charmful modes, for which similar difficulties can 
be apprehended. 

We conclude that Ref. [l[ does not abrogate our 
criticism, because the only known examples where the 
Bayesian treatment leads to a numerically reasonable re- 
sult do not provide any general argument in favor of the 
underlying hypotheses. In contrast, the frequentist ap- 
proach can handle many types of situations smoothly, in- 
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dependently of whether there are free parameters and/or 
theoretical uncertainties. 



II. THE ORIGINAL ISOSPIN ANALYSIS 

The term "isospin analysis" (which in Ref. pj is re- 
placed by "minimal assumptions" ) refers to the original 
paper by Gronau and London Within this frame- 
work, only isospin symmetry is used to parametrize the 
decay amplitudes. Although it is not an exact symmetry, 
its accuracy in B — > tttt is expected to be of the order 
of 1% [5j (once the known dominant contribution from 
electroweak penguins has been taken into account) and 
is thus of phenomenological relevance. Neither the Stan- 
dard Model nor isospin symmetry imply anything on the 
"natural" scale of the hadronic amplitudes. 1 

In Ref. we showed that the Pivk-Le Diberder (PLD) 
and Explicit Solution (ES) parameterizations give the 
most reasonable answer for the Bayesian treatment, be- 
cause the corresponding parameters are close or identi- 
cal to the measured quantities. This is confirmed (for 
the ES parameterization) by the authors of Ref. [l| , who 
thus agree on the fact that other parameterizations lead 
to unstable results. The better behavior of the ES de- 
scription (which follows from our detailed analysis and 
should be acknowledged as such) can be easily under- 
stood as follows. Let us stick to the Gaussian case and 
first assume that there is no degeneracy so that the angle 
a is given by a known single-valued function of the ob- 
servables. One can construct a sampling distribution for 
a as defined by its analytical expression by randomizing 
the observables according to Gaussian PDFs where the 
true (unknown) central values and standard deviations 
have been replaced by their corresponding measured es- 
timators. It is then straightforward to show that the po- 
sition of the peak and its width are consistent estimators 
of, respectively, a and its uncertainty. This procedure 
does not need to define priors nor integration on the true 
values of the parameters, and does not rely on Bayes' 
theorem: it just follows from standard rules of classical 
statistics. 

Technically this procedure can be reproduced by ap- 
plying the "Bayesian" treatment of Ref. @ to the ES pa- 
rameterization with uniform priors. Because for Gaus- 
sian measurements the standard deviation of a quantity 
is equal to the 68% CL frequentist error, it is not a sur- 
prise to have similar numbers in the ES parameteriza- 
tion between the "Bayesian" and frequentist treatments. 
The remaining differences, namely the fact that one finds 
exactly degenerate mirror solutions only in the frequen- 



Note in passing that isospin symmetry was historically found 
before the development of QCD as the theory of the strong in- 
teractions, and the associated notion of a fundamental hadronic 
scale. 



tist approach, mainly come from the integration (logical 
AND) over mirror solutions in the randomization treat- 
ment, which results from an unacceptable interpretation 
of physical constants that could simultaneously take dif- 
ferent values [2|. 

Besides, the Explicit Solution trick is not general 
enough to treat with "minimal assumptions" all the in- 
teresting physical problems. Even when there is the same 
number of parameters as the number of independent ob- 
servables, there is no guarantee that the theoretical equa- 
tions can be inverted to express the parameters as func- 
tions of the observables. The Gronau-London isospin 
analysis is quite special in this respect. More impor- 
tantly, most of the time one has overconstrained systems, 
that is one has more observables than parameters. In this 
case no "Explicit Solution" parameterization exists, and 
no general probability argument would help to find what 
is the most "natural" parameterization for a Bayesian 
treatment with uniform priors. 

To close this Section we point out two misconceptions 
in Ref. The first one is related to the claim that 
the 68% or 95% confidence level or credible intervals are 
more meaningful than the full curve. The fact that ex- 
perimental measurements are often summarized as one- 
or two standard deviation intervals is just a matter of 
convention and if the curve has a complicated shape, it 
cannot be reduced to a one or two-number description 
without loss of information. 2 In particular for the present 
case, we emphasize the importance of having exactly de- 
generate peaks in the angle a (as are reproduced by the 
constrained frequentist fit, see below) to be in agreement 
with isospin symmetry no matter the values of the 68% 
or 95% error intervals. 

The second misconception is related to the above mis- 
understanding of the meaning of the degeneracy that is 
intrinsic to the isospin analysis. The fact that with the 
new data from Summer 2006 [8( one sees only four peaks 
instead of eight as a function of a is not related to the 
improved measurements, but to the fact that now the ex- 
perimental central values are slightly outside the physical 
region that is defined by the isospin symmetry (which is 
related to the property that both the three amplitudes 
and their CP conjugates must form a triangle in the com- 
plex plane). Most presumably the data will evolve in such 
a way that the eightfold ambiguity reappears, which is 
the general case in the mathematical sense, and not a 
"fortuitous accident" as claimed in [l| . Would a different 



"Posterior probability distributions provide the full description 
of our state of knowledge about the value of the quantity. In 
fact, they allow us to calculate all probability intervals of in- 
terest. Such intervals are also called credible intervals. [...J we 
emphasize that the full answer is given by the posterior distribu- 
tion, and reporting only these summaries in the case of complex 
distributions (e.g. multimodal and/or asymmetrical PDFs) can 
be misleading, because people tend to think of a Gaussian model 
if no further information is provided." (G. d'Agostini, 2003 0)- 
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ambiguity pattern be confirmed by better data, then ei- 
ther the Standard Model or the isospin symmetry should 
be questioned. 

III. ADDING EXTERNAL INFORMATION 

The authors of Ref. |l| recommend that the "default" 
plot summarizing the present constraints on the angle 
a coming from B — > 7T7T decays should take into account 
additional theoretical and phenomenological information. 
The physical arguments that are presented in favor of 
such an approach are perfectly legitimate and we do not 
object them. However the problem that was studied in 
detail in [j| is not this one, but rather the original isospin 
analysis. Let us recall the advantages of performing the 
analysis assuming only isospin symmetry. 

• Neither isospin symmetry nor B — > tttt experimen- 
tal measurements give any insight on the typical 
scale of the hadronic amplitudes; thus it is a nat- 
ural choice for the experimental collaborations to 
present their results in a way that is independent of 
possible assumptions on the hadronic amplitudes. 

• The original paper Q only assumed isospin sym- 
metry. 

• One can think of many relevant physical problems 
containing completely free unknown parameters, 
the typical scale of which is not controlled even as 
an order of magnitude. This is the case, e.g., of gen- 
eral Beyond the Standard Model scenarios where 
one parametrizes new arbitrary contributions with 
a few quantities that a priori can take any value. 
The original isospin analysis is thus a pedagogical 
example that exhibit many features that would ap- 
pear in more general situations. 

Having this in mind we can now examine the new pro- 
posal of Ref. First we remark that the statement 
made in [l[ according to which one can get new con- 
straints on a, and eventually lift the degeneracies and 
suppress the solution at a — ► 0, provided one uses ad- 
ditional theoretical and phenomenological knowledge, is 
obviously trivial and was the guideline of, e.g., Section 
VI. 1 of Ref. [§] and many similar studies in the past. 

Second we stress that additional information is ill- 
defined. The assumptions made in [l[ are equivalent to 
an analysis with a finite theoretical error on each pa- 
rameter (except the one that is scanned, here the angle 
a) , such as the "historical" determination of the Unitar- 
ity Triangle [6]. It is well known that in general there 
is no unambiguous definition of the meaning of a theo- 
retical error, and when such uncertainties are present the 
Bayesian and frequentist methods cannot be compared in 
a rigorous way. The authors of Ref. [l[ take as a strong 
argument in favor of their approach the fact that their 
results are weakly dependent of the assumed order of 



TABLE I: For each of the peaks in the plots of Fig. \T\ are 
shown the corresponding values for the parameters as found 
by direct analytical calculation or by the numerical fit. The 
first six solutions correspond to the upper plot, while the three 
others correspond to the lower plot 
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magnitude for the amplitudes. We do not find this con- 
vincing, but rather would like to take the following two 
limits, which to our knowledge are the only ones where 
the concept of a theoretical error becomes perfectly well 
defined. 

• Theory errors go to zero, i.e. the corresponding 
parameters are fixed constants. In this case it is 
trivial to show that both frequentist and Bayesian 
approaches lead to equivalent numerical results. 

• Theory errors go to infinity, i.e. the corresponding 
parameters are completely free unknowns; this is 
the case that is discussed in the previous section 
and where the Bayesian treatment in a generic pa- 
rameterization simply fails. 

Hence, only in the very specific case where both exper- 
imental and theoretical uncertainties are "sufficiently" 
reduced, and there is no free parameter, the numerical 
comparison of the Bayesian treatment with the frequen- 
tist classical approach may become meaningful, 3 and in 
the Bayesian case one might expect a "reasonable" sta- 
bility with respect to priors and parameterization. This 
stability, however, must be extensively checked case-by- 
case, which has not been done in the previous publica- 
tions by the UTfit collaboration Q. 

Fig. Q] shows the results of a constrained frequentist 
fit that takes into account the same information on the 
hadronic parameters as in Fig. 5 of [l[ , namely that the 
amplitudes verify the following bounds: \T^ | < 10, 
T 00 | < 10 and \P\ < 2.5 (in natural units). In particular 
these bounds suppress the parameter configurations with 



3 Trying to validate the Bayesian treatment by numerical com- 
parison with the frequentist approach, without any internal con- 
sistency check, remains a quite peculiar working model. 
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FIG. 1: Frequentist confidence level as a function of the angle 
a, taking into account the following constraints pj: < 
10, |T 00 | < 10 and \P\ < 2.5 (natural units), for B -> tvtt 
decays. Upper plot: data that correspond to the plots in 
Ref. 0|, Lower plot: new data that correspond to Fig. 5 in 



formulas or numerically from the fit. 5 Since these param- 
eter configurations all lead to the very same observables 
and all verify the phenomenological bounds advocated in 
Ref. [![, there is no way to tell which configuration is 
"more probable" . In turn, this means that the Bayesian 
treatment lifts the degeneracy only because of the un- 
physical marginalization over the hadronic parameters, 
and that reducing the output information to the 68% or 
95% probability intervals is very misleading. From Ta- 
ble [J one sees that to lift the degeneracy between the 
remaining solutions, even partially, one would need to 
know the amplitudes with a relative uncertainty of bet- 
ter than 100%, which is feasible using more dynamical 
approaches to hadronic elements, but which would re- 
quest far more than just isospin symmetry and orders of 
magnitude for the hadronic amplitudes. 

To close this section we emphasize that there are many 
interesting problems depending on completely unknown 
parameters on which we have no clue, not even a rough 
order of magnitude. This is for example the case in 
generic new physics scenarios, such as the one of the 
second paper of Ref. There arbitrary new physics 
contributions to BB mixing is parametrized by a quan- 
tity Cb exp(2i</>s); by definition, since no assumption is 
made on specific new physics models, the analysis does 
not give any insight to the "natural" scale of the param- 
eter Cb- Thus one again may expect the Bayesian treat- 
ment to suffer from instabilities when moving from, e.g, 
the (Cb,4>b) to the (xb = cb cos 0b, Vb = cb sin 0b) 
parameterization (such instabilities are expected to in- 
crease with the number of new physics parameters). In 
other words the Bayesian approach as recommended by 
Ref. [l[ cannot be applied (although in practice it is) to 
the full generality of problems at hand. 
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a — > 0, as emphasized in [l|. 4 Indeed this curve is not too 
different from the one that corresponds to the Bayesian 
treatment, however it appears clearly that the frequen- 
tist analysis, in contrast to the Bayesian one, respects the 
exact degeneracy between the mirror solutions that sur- 
vive the phenomenological bounds, as requested by the 
symmetries of the problem. Table U shows the param- 
eter values that correspond to each of the peaks in the 
plots of Fig. [TJ as obtained analytically from the exact 
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4 Without these bounds, arbitrary small, but finite, values for a 
have a good confidence level because large, but finite, values for 
the hadronic amplitudes can generate the observed CP violation. 
However the CP-conserving value a = is strongly disfavored in 
the Standard Model, so that the confidence level is discontinuous 
at this point, as explained in Q. Contrary to the claims in [lj, 
these results are in perfect agreement with quantum field theory 
and do not violate any fundamental principle. 



5 Using new data (lower plot in Fig. [TJ, the exact degeneracy of 
the constrained fit is only threefold because the central values 
are outside the physical region; again the parameter values that 
correspond to the peaks exactly reproduce the same observable 
values, and all satisfy the phenomenological bounds on the am- 
plitudes, so that there is no possibility to tell which solution is 
to be preferred. 
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